Functional approximations to posterior densities: a neural network approach to efficient sampling

The performance of Monte Carlo integration methods like importance sampling or Markov Chain Monte Carlo procedures greatly depends on the choice of the importance or candidate density. Usually, such a density has to be "close" to the target density in order to yield numerically accurate results with efficient sampling. Neural networks seem to be natural importance or candidate densities, as they have a universal approximation property and are easy to sample from. That is, conditional upon the specification of the neural network, sampling can be done either directly or using a Gibbs sampling technique, possibly using auxiliary variables. A key step in the proposed class of methods is the construction of a neural network that approximates the target density accurately. The methods are tested on a set of illustrative models which include a mixture of normal distributions, a Bayesian instrumental variable regression problem with weak instruments and near-identification, and two-regime growth model for US recessions and expansions. These examples involve experiments with non-standard, non-elliptical posterior distributions. The results indicate the feasibility of the neural network approach.Markov chain Monte Carlo;Bayesian inference;importance sampling;neural networks

Neural network approximations to posterior densities: an analytical approach

In Hoogerheide, Kaashoek and Van Dijk (2002) the class of neural networksampling methods is introduced to sample from a target (posterior)distribution that may be multi-modal or skew, or exhibit strong correlationamong the parameters. In these methods the neural network is used as animportance function in IS or as a candidate density in MH. In this note wesuggest an analytical approach to estimate the moments of a certain (target)distribution, where `analytical' refers to the fact that no samplingalgorithm like MH or IS is needed.We show an example in which our analyticalapproach is feasible, even in a case where a `standard' Gibbs approach wouldfail or be extremely slow.Markov chain Monte Carlo;Bayesian inference;importance sampling;neural networks

On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: an application of flexible sampling methods using neural networks

Likelihoods and posteriors of instrumental variable regression models with strongendogeneity and/or weak instruments may exhibit rather non-elliptical contours inthe parameter space. This may seriously affect inference based on Bayesian crediblesets. When approximating such contours using Monte Carlo integration methods likeimportance sampling or Markov chain Monte Carlo procedures the speed of the algorithmand the quality of the results greatly depend on the choice of the importance orcandidate density. Such a density has to be `close' to the target density in order toyield accurate results with numerically efficient sampling. For this purpose we introduce neural networks which seem to be natural importance or candidate densities, as they have a universal approximation property and are easy to sample from.A key step in the proposed class of methods is the construction of a neural network that approximates the target density accurately. The methods are tested on a set ofillustrative models. The results indicate the feasibility of the neural networkapproach.Markov chain Monte Carlo;Bayesian inference;credible sets;importance sampling;instrumental variables;neural networks;reduced rank

"Rotterdam Econometrics": an analysis of publications of the econometric institute 1956-2004

The high ranking of the Econometric Institute,as listed in recent leading scientific journals, is examinedfor a fifty year period using similar standard measures. The distribution of the publicationsover different research areas is analyzed and a time-series model is specified to describeand forecast the publication pattern.

Neural network based approximations to posterior densities: a class of flexible sampling methods with applications to reduced rank models

Likelihoods and posteriors of econometric models with strong endogeneity and weakinstruments may exhibit rather non-elliptical contours in the parameter space.This feature also holds for cointegration models when near non-stationarity occursand determining the number of cointegrating relations is a nontrivial issue, and in mixture processes where the modes are relatively far apart. The performance ofMonte Carlo integration methods like importance sampling or Markov ChainMonte Carlo procedures greatly depends in all these cases on the choice of the importance or candidate density. Such a density has to be `close' to the targetdensity in order to yield numerically accurate results with efficient sampling. Neural networks seem to be natural importance or candidate densities, as they havea universal approximation property and are easy to sample from. That is, conditionallyupon the specification of the neural network, sampling can be done either directly orusing a Gibbs sampling technique, possibly using auxiliary variables. A key step in the proposed class of methods is the construction of a neural network that approximatesthe target density accurately. The methods are tested on a set of illustrative modelswhich include a mixture of normal distributions, a Bayesian instrumental variable regression problem with weak instruments and near non-identification, a cointegrationmodel with near non-stationarity and a two-regime growth model for US recessionsand expansions. These examples involve experiments with non-standard, non-ellipticalposterior distributions. The results indicate the feasibility of theneural network approach.Markov chain Monte Carlo;Bayesian inference;neural networks;importance sample

"Rotterdam econometrics": publications of the econometric institute 1956-2005

This paper contains a list of all publications over the period 1956-2005, as reported in the Rotterdam Econometric Institute Reprint series during 1957-2005.

A bilinear programming solution to the quadratic assignment problem

The quadratic assignment problem (QAP) or maximum acyclical graph problem is well documented (see e.g. Pardalos and Wolkowicz, 1994). One of the authors has published some material, in which it was tried, by structuring the problem additionally, to bring it as closely as possible in the neighbourhood of a binary solution (seePaelinck, 1983, pp. 251-256 and 273-277); good but not optimal solutions could so be obtained (see Paelinck, 1985, pp. 247-254). The problem is taken up again here, in the same spirit but at the same time in a different vein.maximal acyclical graph;quadratic assignment problem QAP

Neural networks as econometric tool

The flexibility of neural networks to handle complex data patterns of economic variables is well known. In this survey we present a brief introduction to a neural network and focus on two aspects of its flexibility . First, a neural network is used to recover the dynamic properties of a nonlinear system, in particular, its stability by making use of the Lyapunov exponent. Second, a two-stage network is introduced where the usual nonlinear model is combined with time transitions, which may be handled by neural networks. The connection with time-varying smooth transition models is indicated. The procedures areillustrated using three examples: a structurally unstable chaotic model, nonlinear trends in real exchange rates and a time-varying Phillips curve using US data from 1960-1997.Exchange rates;Neural networks;Nonlinear dynamics;Phillips curve

A bilinear programming solution to the quadratic assignment problem

The quadratic assignment problem (QAP) or maximum acyclical graph problem is well documented (see e.g. Pardalos and Wolkowicz, 1994). One of the authors has published some material, in which it was tried, by structuring the problem additionally, to bring it as closely as possible in the neighbourhood of a binary solution (see Paelinck, 1983, pp. 251-256 and 273-277); good but not optimal solutions could so be obtained (see Paelinck, 1985, pp. 247-254). The problem is taken up again here, in the same spirit but at the same time in a different vein